Papers with differential privacy based training approaches
Fine-Tuning Language Models with Differential Privacy through Adaptive Noise Allocation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Language models can memorize detailed information and patterns, but raise privacy concerns . ANADP reduces the performance gap between regular and DP fine-tuning while maintaining the privacy constraints. |
| Approach: | They propose an algorithm that allocates additive noise based on the importance of model parameters to reduce the performance gap between regular fine-tuning and traditional DP fine- tuning. |
| Outcome: | The proposed algorithm narrows the performance gap between regular fine-tuning and traditional DP fine- tuning while maintaining privacy constraints. |